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Abstract 


Current plans for independent instrument approaches to closely spaced parallel 
runways call for an automated pilot alerting system to ensure separation of aircraft in the 
case of a “blunder,” or unexpected deviation from the normal approach path. Resolution 
advisories by this system would require the pilot of an endangered aircraft to perform a 
trained evasion maneuver. 

The potential performance of two evasion maneuvers, referred to as the “turn-climb” 
and “climb-only,” was estimated using an experimental NASA alerting logic (AILS) and a 
computer simulation of relative trajectory scenarios between two aircraft. One aircraft was 
equipped with the NASA alerting system, and maneuvered accordingly. Observation of the 
rates of different types of alerting failure allowed judgement of evasion maneuver 
performance. System Operating Characteristic (SOC) curves were used to assess the 
benefit of alerting with each maneuver. 

This analysis shows the climb-only maneuver to be a poor substitute for the turn- 
climb. For a 2500 ft runway spacing and an expected 2 sec pilot reaction time, and with the 
nominal alerting threshold settings chosen by NASA for the turn-climb, false alarms during 
blunders are approximately 40 times as likely to induce collisions when using the climb- 
only as when using the tum-climb, and 40 times as many collisions occur during blunders 
with the climb-only overall. SOC analysis shows that the safety possible with the climb- 
only is difficult to distinguish from having no alerting system at all. With the tum-climb 
there is a clear safety benefit. Alerting performance with the tum-climb is also more 
resistant to errors in trajectory prediction and evasion maneuver execution. 


This document is based on the thesis of Lee F. Winder submitted in partial fulfillment of 
the degree of Master of Science in Aeronautics and Astronautics at the Massachusetts 
Institute of Technology. 
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Chapter 1 


Introduction 


1.1 Collision Avoidance for Independent Parallel Approaches 

Parallel runway landing operations are most efficient when the approaches are 
independent (Ebrahimi 1993, PRM Program Office 1991), that is, when aircraft on 
approach to one runway are not constrained in motion by aircraft in an adjacent queue. 
Because of the danger of a mid-air collision, approaches to parallel runways may be 
independent only in special circumstances: either visual meteorological conditions (VMC) 
must exist, so that pilots can see traffic, or traffic must be monitored by another party that is 
able to intervene with warnings during a conflict. In the latter case, runway separation must 
also be greater than some minimum. This is to compensate for delays associated with the 
surveillance and alerting mechanism. The most advanced parallel approach warning system 
currently available, the Precision Runway Monitor (PRM), may operate at runway 
separations down to 3400 ft. Numerous airports have runways spaced more closely than 
this, and are thus unable to carry out independent approaches during instrument 
meteorological conditions (IMC). 

Automated airborne alerting, in conjunction with new and more precise methods of 
approach guidance, may enable a reduction in the minimum runway spacing for independent 
instrument approaches. Rather than give air traffic control (ATQ sole responsibility for 
surveillance and alerting, as is the current practice, automated alerting requires placement of 
a computerized monitoring and alert system aboard each aircraft. Arguments in favor of 
automated alerting include reduced delay in issuing warnings, and elimination of missed or 
late alerts due to human error. 
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1.2 Climb-Only vs. Turn-Climb Evasion Maneuver 


The proposed alerting system accepts measurements of variables that describe the 
state of the aircraft and surrounding environment, using these to estimate the risk of a 
collision. If a collision is likely, an alert is issued via cockpit displays in time for the pilot to 
escape using a particular evasion maneuver. 

To ensure that evasion maneuvers are carried out with adequate promptness and 
precision, it has been assumed that the maneuver will be a fixed procedure for which pilots 
can train in advance. Because the parallel approach takes place at low speed, near the 
ground, and with traffic on at least one side of each participating aircraft, few reasonable 
maneuvering options exist. The procedure that has been assumed most often in the past, 
and is standard with the Precision Runway Monitor, is referred to in this report as a turn- 
climb. It requires a coordinated turn to a specific heading, and a simultaneous pull-up to a 
certain final vertical speed. A maneuver option of more recent interest is a climb-only , where 
the aircraft follows the runway centerline while accelerating vertically. This is a simpler 
maneuver for the pilot to execute, resembles a standard missed approach maneuver, and may 
allow easier handling of the evading aircraft by air traffic controllers than does the tum- 
climb. While the turn-climb has benefits of high performance (in terms of total impulse), 
using it may require special training of both pilots and air traffic controllers, leading to 
objectionable costs. The climb-only avoids or reduces these problems, but sacrifices 
performance. 

1.3 Thesis Overview 

This thesis presents an analysis to determine whether a climb-only maneuver provides 
performance sufficient that it can be substituted for the original tum-climb. The issue is 
studied in the context of an alerting logic now under consideration by the NASA Langley 
Research Center. The approach was to simulate a large number of trajectories of an 
intruding aircraft relative to a host aircraft equipped with the NASA alerting system. 
Scenarios were repeated while varying parameters of the alerting threshold and evasion 
maneuver. The rates of specific alerting failures were noted. Simulation output was given a 
probabilistic interpretation for analysis. System Operating Characteristic (SOC) curves 
were used to view the tradeoff of collisions for false alarms, and the benefit of each 
maneuver, at different system parameter settings. 
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Chapter 2 provides background information concerning the independent IMC parallel 
approach, the development of the NASA logic, and evasion maneuver selection. Chapter 3 
describes the theory of alerting performance necessary to interpret the experimental data, 
and discusses the relationship of alerting system design choices to different types of 
alerting failure. Chapter 4 presents the trajectory simulation, results and interpretation. A 
summary and concluding discussion are in Chapter 5. 
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Chapter 2 


Alerting System Background and AILS Logic 


2.1 Independent Parallel Approaches in Instrument Conditions 

In visual meteorological conditions FAA regulations allow streams of aircraft to 
approach parallel runways independently. During independent parallel approaches each 
runway is operated as though no other runway were nearby, specifically in that no hard 
longitudinal position constraints are imposed between aircraft in parallel streams. Parallel 
runways operate at maximum capacity when approaches are independent. During a 
simultaneous independent VMC approach, pilots have responsibility to see and avoid one 
another. The VMC approach can be carried out on runways separated by as little as 700 ft 
(Ebrahimi 1993, PRM Program Office 1991), though wake vortices must be considered. 

In instrument meteorological conditions pilots are able to carry out final approach 
using cockpit displays, but may be unable to see either the runway or other aircraft. The 
loss of awareness is significant enough that, with existing cockpit displays, pilots are not 
able to assume ultimate responsibility for separation assurance. To maintain safety, 
additional safeguards are introduced during independent IMC approaches. 
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Figure 2.1: Parallel Approaches in Instrument Conditions 
Using ATC Surveillance 

The traditional method of safeguarding against collisions during independent IMC 
approaches has been to use ground-based surveillance and intervention (Ebrahimi 1993, 
PRM Program Office 1991). In this method, illustrated in Figure 2.1, air traffic controllers 
monitor streams of aircraft on final approach using radar. If an aircraft deviates 
significantly (termed a blunder) from its expected approach path, it is the responsibility of 
controllers to intervene by issuing radio commands to the pilots involved. For the 
intervention to succeed, adequate separation must exist between the blunderer and any 
endangered aircraft at the time the blunder is detected. The separation needed depends on 
aircraft maneuverability, the speed with which breakout instructions can be issued to pilots 
after a blunder, and pilot reaction time. Because the blunderer may be at any position along 
its approach centerline relative to the endangered aircraft at the start of the blunder, the lower 
bound on initial separation must be set through restrictions on runway separation 1 . With 
conventional ATC surveillance, independent IMC approaches are permitted on parallel 
runways spaced as closely as 4300 ft. Most airports having parallel runways spaced below 
4300 ft suffer a loss of capacity during intervals of poor visibility, because controllers must 
resort to a less productive method of operation, which is discussed in the following section. 


1 It is assumed throughout this paper that final approach takes place along the extended runway 
centerline. Thus, runway separation is the same as approach stream separation. 
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2.2 Dependent Parallel Approaches 

Until recently, instrument approaches at runway spacings below 4300 feet were 
possible only using dependent approaches. By imposing a minimum longitudinal as well as 
lateral separation, the minimum runway separation can be reduced while maintaining an 
acceptable overall separation (Figure 2.2). The current requirement for dependent 
approaches is that a minimum horizontal range of 2 miles be maintained between aircraft in 
adjacent corridors (Ebrahimi 1993, PRM Program Office 1991). 



Independent Approach 


Dependent Approach 


Figure 2.2: Independent and Dependent Approaches 

Whereas independent approaches to a pair of parallel runways result in a total 
capacity about twice that of a single runway, dependent approaches to those same runways 
are typically less productive. One reason is geometric. For the 2 nmi minimum horizontal 
range, the average in-trail spacing during dependent approaches of aircraft in each corridor 
is higher than it would be if the aircraft were on independent approaches, resulting in a 
lower landing rate (PRM Program Office 1991). The dependent approach is also more of a 
challenge for air traffic controllers to operate than the independent approach. For their own 
comfort, controllers may maintain a horizontal separation greater than required, further 
increasing the average in-trail spacing (PRM Program Office 1991). Finally, while 
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independent approaches allow air traffic control to separate fast and slow (jet vs. propeller) 
traffic into parallel streams without penalty, dependent approaches force all traffic to adhere 
to the slower speed, or otherwise increase the difficulty and inconvenience of the process. It 
has been shown that independent approaches are about 30% more productive than 
dependent approaches to a given pair of runways (Ebrahimi 1993). 

2.3 The Precision Runway Monitor 

Due to the performance disparity between the dependent and independent approach 
methods, independent approaches are preferred where possible. There is interest, 
consequently, in enabling independent approaches at runway spacings below the usual 4300 
ft minimum. The Precision Runway Monitor (PRM) was developed for this purpose. For 
addressing some of the weaknesses of conventional radar surveillance, PRM has been 
granted approval for use at down to 3400 ft runway spacings (Shank & Hollister 1994). 

Inherent in the conventional terminal air surveillance system are uncertainties and time 
lags that limit the precision with which controllers can direct traffic. An approach 
controller’s radar display updates about once every 4.8 seconds. Interpretation of discrete 
radar data involves a definite lag, and even a major course change by a monitored aircraft 
may be not be detectable until several seconds after the event Delays due to controller 
reaction time in detecting an abnormal event and delays inherent in voice communication 
further lengthen the interval between the beginning of a blunder and initiation of the 
resolution. Any source of delay increases the runway spacing required for reliable 
prevention of collisions. 

The Precision Runway Monitor is a surveillance and decision aid that operates in 
parallel with conventional approach air traffic control, supplementing it with relatively high 
performance radar and display technology. Specialized monitoring personnel watch traffic 
on high resolution CRT displays, whose information is provided by enhanced radar that is 
more precise (1 vs. 5 milliradian) and is updated more frequendy (at most 2.4 s vs. 4.8 s) 
than regular terminal radar (Ebrahimi 1993, PRM Program Office 1991). The PRM 
operators do not interact with pilots during normal approach operations, but are expected to 
intervene under the abnormal circumstance of a blunder. If a blunder occurs, the PRM 
operators use the control tower communication frequency to issue corrective commands to 
pilots. 
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2000 ft or more 


Figure 2.3: Schematic of Precision Runway Monitor Display 

To aid monitors in the timely detection of blunders, an automated alerting system 
capable of generating both visual and aural alerts is built into the PRM display (Lind 1993), 
which is shown schematically in Figure 2.3. Alerts occur if monitored aircraft cross into or 
are predicted to cross into an at least 2000 foot wide No Transgression Zone (NTZ) 
separating each pair of runways. Trajectory predictions are based on an assumption that 
aircraft travel at constant velocity. An aircraft predicted to enter the NTZ within a limited 
time interval (zero to ten seconds, as chosen by the PRM operators) triggers a caution alert. 
An aircraft actually entering the NTZ triggers a warning, indicating that evasive maneuvers 
are necessary. Any endangered aircraft on the opposite side of the NTZ is issued a verbal 
tum-climb breakout command. Though the operator will almost certainly take action by the 
time an NTZ crossing takes place, the precise moment to intervene is a subjective choice 
made by the operator, and the PRM alerts serve more as decision aids than hard commands 
(Lind 1993). 
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2.4 Automated Cockpit Alerting 


Through refined radar surveillance, PRM enables a reduction in the minimum runway 
spacing, but it is not free of all of the delays in traditional air traffic control methods. Of the 
remaining delays, many are inherent in the use of ground-based human approach monitors, 
and therefore can not be eliminated through any improvement in radar or display 
technology. A recent proposal has been to supplement ground-based human monitoring 
with automated cockpit-based alerting systems similar to the Traffic Alert and Collision 
Avoidance System (TCAS) and Ground Proximity Warning System (GPWS). 

Automated cockpit alerting promises to reduce alerting delays that might otherwise 
occur. Human monitors are subject to unpredictable and often long delays, particularly 
when the event of concern, a blunder, rarely occurs. An automated system will also be more 
consistent in the decisions it makes, and may be capable of alerting decisions closer to 
optimal than those provided by human monitors. 

Though not limited to systems using automated decision making, there may also be a 
benefit to using specialized alerting displays instead of voice commands to pilots. A 
problem with current methods is that controllers must devise and communicate instructions 
for evasion maneuvers by radio while the blunder is in progress. A complex message may 
result Proposed automated alerting systems specify that the evasion maneuver be planned 
and trained for in advance, so that only a simple stimulus is required. 

TCAS is an automatic cockpit alerting system designed to reduce the incidence of 
mid-air collisions by aircraft en route and in the terminal area. A computer aboard each 
TCAS aircraft obtains situation variables from on-board instruments, through beacon radar 
surveillance, and via limited datalink from nearby aircraft. Using this information the 
computer judges the level of risk, and selects an action from a range of options. The 
alerting system may do nothing if risk is low, issue a caution advisory if risk is moderate, or 
issue a resolution advisory with maneuver commands (only with the most advanced version 
of TCAS) if a collision seems imminent. Alerts are displayed visually and aurally to the 
pilot, and a visual command display aids in execution of evasion maneuvers. TCAS alerts 
that are generated in different aircraft are coordinated so that commanded evasion 
maneuvers do not induce a collision. 

The success of TCAS gives credibility to plans for a similar system to prevent 
collisions between aircraft on independent parallel approach. Such an alerting system can 
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not be a trivial extension of TCAS. It was decided early in AILS development that 
automated parallel approach alerting will require relatively complete state information, 
including enough state variables for accurate projection of trajectories over a limited interval 
(Koczo 1996). The alerting thresholds of TCAS use only three dimensional range, the time 
derivative of this range, relative altitude, and the time derivative of relative altitude. Thus, 
TCAS is able to function without the initial relative velocity of the aircraft, which along with 
relative position is the minimum information that will allow an explicit single trajectory 
model which is correct in the near term. TCAS thresholds are not derived from any specific 
trajectory model. They are an empirical result obtained through iterated evaluation and 
adjustment of a baseline structure. Attempts to adapt the TCAS thresholds to parallel 
approach alerting through parameter adjustment result in a system that, to detect blunders 
early enough, must alert frequently during normal approaches (Folmar et al. 1994, Koczo 
1996, Toma & Massimini 1993). Proposed alternatives to an adapted TCAS have used a 
large set of state variables, including velocity, turn rate, and GPS-derived position, to 
explicitly model the future trajectories of aircraft in three dimensional space. Turn rate, 
though unnecessary for a simple (constant velocity) trajectory prediction, provides lateral 
acceleration information, and may improve prediction for short intervals. 

Whereas TCAS is able to choose from a range of evasion maneuvers, for parallel 
approach alerting systems it has typically been assumed that there is a single evasion 
maneuver option, and that the pilot trains with this maneuver in preparation for possible 
alerts. This method of alerting should result in the shortest pilot reaction tune, allowing 
alerts to take place late into the blunder, and minimizing false alarms. Better performance is 
conceivable if the alerting system were free to choose from a range of maneuver possibilities 
as TCAS does, but the additional difficulty of interpreting the more complex alerts might 
increase reaction time unacceptably. Also, there is comparatively little room for maneuver 
variation for an aircraft in slow flight near the ground and blocked on one side by an 
intruder, meaning the additional complexity of multiple maneuvers might not pay off even if 
reaction time were not an issue. 

One concept for a parallel approach alerting logic was developed at MIT’s 
Aeronautical Systems Laboratory (Kuchar & Carpenter 1997). The alert thresholds were in 
terms of the probability of a collision during execution of an escape maneuver. If an 
estimate of this probability exceeded a chosen value, an alert was issued. The goal was to 
minimize alerts while maintaining a specified acceptable collision risk. In parallel to MIT’s 
work, several logics were proposed and evaluated by Rockwell-Collins for the NASA 
Langley Research Center (Koczo 1996). These logics employed a variety of methods of 
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extrapolating aircraft position trajectories into the future from the measured initial state. A 
modification of one of the Rockwell logics has been adopted as the primary candidate for 
use in the Airborne Information for Lateral Spacing (AILS) system. It is referred to, from 
here on, as the AILS logic, or simply ALLS. 

What follows in section 2.5 is a description of aspects of the AILS logic relevant to 
this research, as of early 1998. The AILS system is under continuing development by 
NASA, so the structural and performance information reported here may no longer be 
accurate. 

2.5 NASA Airborne Information for Lateral Spacing Logic 

The AILS system takes as inputs state variables for the host aircraft, and for any other 
aircraft nearby. In addition, the system must know the geometry of the host’s approach 
path. Information not available aboard the host itself is datalinked by other aircraft. The 
AILS computer is able to generate six distinct alerts (Table 2.1) to the pilot of the host 
aircraft. The input-output structure is shown in Figure 2.4. 



Alerts can be divided into two categories, each corresponding to a distinct logic 
(Waller & Scanlon 1996). The first type is based on adherence of the host aircraft (the 
aircraft on which the described alerting system is located) to acceptable trajectory states. 
During a normal approach, the host should follow a predictable path along the runway 
centerline. Within 10 nmi of the runway threshold and prior to the middle marker, this path 
is enclosed by two concentric constant-width corridors (Figure 2.5). If the host aircraft 
deviates sufficiently from the expected path to leave the inner corridor, a caution alert is 
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issued to its pilot. If it then leaves the outer corridor, a breakout alert is issued. Beyond 10 
nmi the corridors widen to accommodate the host during initial capture of the localizer 
(actually, a virtual localizer, since the approaches are to be based on GPS navigation). After 
the middle marker, the corridors taper to a point in the same manner as a localizer beam. 
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Figure 2.5: 
The second type 


Corridors for AILS Approach Conformance Alerts 

of alert involves extrapolation of the states of all aircraft into the 


future. Given the initial state of the system, a range of potential trajectories is considered. If 
any of these trajectories brings another aircraft into a defined hazard zone about the host, 
within a limited time, an alert is issued. The hazard zone is a cylinder of height 2H and 


radius R centered about the host. The trajectory extrapolation model includes potential 
blunders by both the host and other aircraft. 


The sequence of testing is as follows. 

First the host is modeled as the blunderer, and the other aircraft as on a normal 
approach. If it is possible for the host to enter another aircraft’s hazard zone within a tune 
T, an alert is issued. This condition is checked simultaneously against two sets of the 
parameters R, H and T, corresponding to two alerting thresholds. The parameter values are 
chosen so that the thresholds will be crossed in a definite sequence as the blunder 
progresses. The first alert to occur is a caution and the second a breakout command. 
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Possible blunder trajectories 
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Table 2.1: AILS Alerts 



Stage 1 

Stage 2 

Logic 1: Host Approach Conformance Alert 

Caution 

Breakout 

Logic 2: Collision Predicted 

Host Blundered 

Caution 

Breakout 

Other Aircraft Blundered 

Caution 

Breakout 


This process is repeated for modeled trajectories in which the host is assumed to be 
on a correct approach, and the other aircraft to be blundering. Once again there are two 
thresholds, corresponding to a caution and a breakout alert aboard the host. The four 
extrapolation alert thresholds are designed so that the blundering aircraft will receive each 
type of alert (caution and breakout) before it is issued to the normally approaching aircraft. 

The philosophy of AILS requires that an individual pilot be able to ensure the safety 
of his own aircraft in situations where an approaching blunderer is unable to return to its 
own approach corridor. In view of this, the analysis carried out for this thesis considers a 
reduced version of the AILS trajectory set, illustrated in Figure 2.6. It is simply the subset 
of the full logic that is applicable when it is assumed that the host aircraft is on a correct 
approach, and that an intruding blunderer will fail to respond to any alerts. This is 
presumably the worst case, and is unlikely, due to the number of caution and breakout 
warnings that must occur before reaching such a state. Performance of the system under 
these circumstances should place a lower bound on actual safety. 

If the intruder is assumed not to respond to alerts (e.g., due to flight control failure), 
and the host is on a normal approach course, five out of the six alerts can be neglected. The 
remaining alert is an extrapolative breakout alert (the bottom, right cell in Table 2.1). 

The host is assumed to travel at constant speed along the extended runway centerline 
and vertically along a standard 3° glide slope. The speed and initial position are 
approximated using measured states. As shown in Figure 2.6, the intruder may travel along 
any of several trajectories, ranging from a constant rate turn at the current turn rate to a 
linear trajectory along the current velocity vector. For intermediate trajectories, the intruder 
begins a constant rate turn but rolls out into straight line flight at some point along the turn. 
Airspeed and vertical speed are assumed to be constant. 
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If for any possible intruder trajectory the two aircraft are able to pass within a 
horizontal distance R and simultaneously a vertical distance H of one another within T 
seconds from the present, the host is issued a breakout alert. The three constants R, H and 
T must be set during design of the logic so that long term performance of the alerting 
system is acceptable. At NASA Langley Research Center this was done by subjecting a 
simulated AILS host to a variety of hazardous and non-hazardous trajectory scenarios using 
piloted simulators, having the host respond to alerts with a particular evasion maneuver, and 
noting the rates of various failures as the parameters were varied. For a 2500 ft runway 
spacing, parameters of R = 550 fit, H = 550 ft, and T = 13 sec were chosen by NASA as 
producing roughly the best overall performance with the assumed evasion maneuver. 

2.6 AILS Evasion Maneuver 

In NASA’s initial tests of the AILS logic the evasion maneuver used was a turn- 
climb, consisting of a 45° change of heading away from the intrader by way of a turn of 30° 
maximum bank, a 0.25 g pull-up to a final 2000 ft/min climb rate, and a 1 kt per second 
acceleration to a 15 kt final airspeed increase. Pilot reaction times were typically below 2 
seconds (Waller & Scanlon 1996). This is a similar maneuver to that expected of pilots 
using PRM. Using AILS with the tum-climb evasion, the NASA researchers were able to 
achieve satisfactory alerting performance. 

It was later suggested that a straight-ahead climb, or climb-only, performed along the 
horizontal approach centerline, would be a preferable form of evasion. Such a maneuver 
resembles a standard missed approach procedure and thus might be executed more reliably 
and promptly by pilots, and with less training expense. It may also be simpler from an air 
traffic control point of view. An aircraft making a 45° heading change on short notice in 
terminal airspace is more likely to interfere with other aircraft than one performing a climb- 
only maneuver. Unfortunately, the climb-only maneuver provides less acceleration and total 
impulse than a tum-climb, probably necessitating earlier detection of blunders, and 
increasing the rate of false alarms. The alerting threshold settings adequate for the tum- 
climb maneuver would be incorrect for the climb-only, and it is unclear whether adequate 
performance is obtainable with a climb-only maneuver through any simple adjustment of the 
thresholds*. 


Even if simple adjustments fail, a complete redesign of the logic could help if the new logic were to 
take advantage of blunder properties currently unknown. 
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For approaches taking place at a 2500 ft runway spacing, the interval between the start 
of a severe blunder and a collision can be on the order of 20 seconds. This leaves little 
room for error in performance of the planned evasion maneuver. A component of any 
evasion maneuver is the interval of pilot latency following an alert, during which the host 
aircraft continues along a normal approach path. If pilots exceed the assumed latency, 
alerting system performance may not meet expectations. The nominal AILS threshold 
parameters reflect reaction times below 2 seconds. Pilots are easily capable of such reaction 
times when recently trained and expecting an alert, as has been the unavoidable case in AILS 
testing, but a pilot who has not experienced a breakout alert in perhaps months or years may 
take longer to react. It is worth noting that the designers of TCAS assumed a more 
generous 5 second reaction time (RTCA 1983). But, TCAS also uses more complex alerts 
covering numerous maneuver options, so the additional seconds may be needed. There are 
currently situations in aviation where a 2 second reaction time assumption is used, typically 
where the pilot is known to be devoting full attention to control (e.g., during takeoff). 
During cruise, with the autopilot engaged, longer reaction times might occur. Because AILS 
is for use during final approach, where full attention is expected, the 2 sec reaction time 
estimate may be justified through argument. For additional insurance, it would be useful to 
show experimentally that 2 seconds is a reasonable delay for this particular application of 
alerting, or to demonstrate that with the chosen evasion maneuver the alerting system is 
insensitive to errors in the estimated reaction time. 

As a preliminary analysis, the relative effectiveness of the two maneuvers was 
compared for a standard imminent collision scenario. With two aircraft initially on final 
approach at 145 kts and constant altitude to parallel runways spaced 2500 ft apart, one was 
made to bank instantaneously to 30°, rolling out at a heading 30° from the approach heading 
so that a collision with the other was imminent. The endangered, or host, aircraft was 
assumed to have a perfect alerting system, able to issue a breakout command at the instant 
the blunder began. Against the blunder, the host aircraft performed one of three maneuvers: 
a tum-climb, a climb-only, and no evasion. The turn -climb evasion was modeled as a 2 sec 
pilot latency followed by (1) a 0.25 g vertical acceleration to a final 2000 ft/min climb, and 


+ This is consistent with analysis done for PRM, in which a 30* heading change toward the adjacent 
runway centerline was considered the “worst case” blunder if uncorrected (Lind 1993, PRM Program Office 
1991). 
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Time From Start of Blunder (sec) 

Figure 2.7: Comparison of Evasion Maneuvers with a 30° Heading Blunder 

simultaneously (2) an instantaneous 30° bank to a final 45° heading change. Airspeed was 
approximately constant. The climb-only evasion was modeled using the latency and climb 
parts of the tum-climb, with lateral position fixed along the runway centerline. 

In Figure 2.7 the separation of points representing the two aircraft is plotted versus 
the number of seconds into the blunder. The blunder results in a collision (a separation 
under 500 ft) in roughly 20 sec for a 2500 ft initial separation when the endangered aircraft 
does not evade. Both evasion maneuvers prevent the collision, but the climb-only maneuver 
does so with significantly less divergence from the non-evasion trajectory. The climb-only 
maneuver results in about a 700 ft closest approach, compared to nearly 2500 ft for the tum- 
climb. 


Superficially, the climb-only maneuver appears a more economical option than the 
tum-climb. But because this is an ideal scenario, the above analysis is misleading. Real 
alerting systems suffer a variety of failures. They may not alert at the instant a blunder 
begins, or only when a collision is imminent. As discussed, the evasion maneuver may also 
vary from the designed form due to reaction time and other types of randomness. The 
chosen evasion maneuver must provide adequate safety even in less-than-ideal 
circumstances. 


25 



2.7 Research Goals and Method 


The turn-climb evasion is believed to provide acceptable performance, but if feasible, 
the climb-only offers certain advantages. The goal of this research is to judge whether the 
climb-only evasion maneuver gives adequate performance for use with the AILS system. 
This requires an understanding of what is meant by adequate performance, and methods of 
measuring performance. The experimental approach was to apply a probabilistic blunder 
model to a simulated aircraft equipped with the AILS logic, and to observe and interpret the 
rates of different types of failures. 
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Chapter 3 


Alerting System Performance 


3,1 Ideal Alerting Performance 

The AILS alerting system provides a discrete alerting stimulus to the pilot of the host 
aircraft when an encounter with another aircraft is predicted. Both cautionary and breakout 
alerts are possible. The pilot understands a breakout alert as a cue to perform a trained 
evasive procedure. 

If an alert is issued by the alerting system, the host aircraft should follow a specific 
evasion trajectory that diverges sufficiently from the non-evasion trajectory to prevent a 
collision (provided that a collision is imminent with the non-evasion trajectory, and that the 
alert is issued early enough). Once a breakout alert occurs, the pilot is committed to 
following through with an evasion, and the function of the alerting system is then, if 
anything, to guide or terminate this maneuver. Each of the trajectory options (evasion or 
non-evasion) will have a distinct outcome, consisting of either a collision, a safe missed 
approach, or a normal landing. The designer of the alerting system hopes to favor some 
outcomes over others by shaping the alerting threshold so that an appropriate trajectory is 
chosen at each instant in time. 

For an aircraft initially on normal approach, the preferred outcome is for the approach 
to finish with a successful landing. If this goal is impossible due to the actions of an 
intruding aircraft, it is better to abort the approach, perform a go-around and temporarily 
disrupt the flow of traffic than to collide with the intruder. A collision is the least desirable 
outcome by far. Assuming this hierarchy of outcomes, an ideal alerting system interferes 
only when a collision would otherwise occur, and effectively replaces a potential collision 
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with a go-around. As discussed below, for a variety of reasons it is impossible for AILS (or 
any other alerting system) to attain this ideal performance in all scenarios. 

3.2 Alerting Failures 

AILS alerting failures fall within two categories: collisions and false alarms. A 
collision occurs when at least one aircraft blunders from its approach path and finally 
collides with another, despite the presence of the alerting system. A false alarm occurs 
when an alert is issued that is not necessary to prevent a collision. These two categories are 
not mutually exclusive, because it is possible for a false alarm to cause a collision (termed 
an induced collision). These possible failures of an alerting system are diagrammed in 
Figure 3.1. Collisions that are not induced collisions are either late alerts or missed 
detections, in which a necessary alert occurs too late or not at all. False alarms that do not 
cause collisions are referred to as unnecessary alerts. Though not catastrophic, 
unnecessary alerts result in needless go-arounds that reduce the effective capacity of the 
runways over time. 


Collisions False Alarms 



Figure 3.1: Alerting System Failures 
3.3 Failure Characteristics of AILS Predictive Alerting Logic 

In this section, properties of the AILS trajectory prediction alerting thresholds are 
related to the different types of alerting failure. 

The AILS alerting system generates alerts based on an explicit projection of aircraft 
trajectories into the future from an observed current state. If a collision or close encounter 
can occur within a limited time and within certain maneuvering limits imposed on an 
intruder, an alert is displayed (visually and/or aurally) to the pilot of the host aircraft. If the 
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trajectory model is correct in its prediction of an encounter, a well chosen evasion maneuver 
will produce a relative acceleration that reduces the chance of a collision. 

To clarify the inherent error tendencies of such a logic it is first useful to consider the 
ideal case, in which it is possible to model the trajectories and dimensions of involved 
aircraft with whatever precision is needed. Given this capability, the alerting system can 
know absolutely whether or not an alert is required well before an accident occurs. If the 
evasion maneuver can be predicted with equal precision, alerts can be delayed until the last 
moment before the evasion becomes unsafe as well, with no penalty on performance. 

The parallel approach system is complex enough that the ideal alerting logic is 
impossible. In modeling the system, errors are unavoidably introduced, and these ensure a 
finite rate of alerting failures. 

Collision Criterion Errors 

Intuitively the word “collision” implies a damaging encounter between aircraft. 
Whether or not a collision occurs depends on the relative position, size, and shape of 
aircraft, and even on aerodynamic interactions between them. In constructing an alerting 
system, it would be difficult to consider all relevant factors for determining a collision, due 
to the number of state variables that would be necessary for the model. In practice, aircraft 
are often modeled as spheres or other simple geometric shapes fixed about the center of 
mass, so that less information is needed for the alerting decision. This is essentially the 
method used by AILS. In exchange for the increased convenience of determining a 
collision by simple criteria, the designer accepts occasional alerting decisions that are 
contrary to the intuitive notion of a collision stated above. This is illustrated in Figure 3.2, 
using two relative trajectory scenarios. These could represent parallel approach scenarios, in 
which a blundering aircraft drifts sideways from right to left toward an aircraft on normal 
approach, missing in one case (a and c), and colliding in the other (b and d). Figure 3.2 (a) 
and (b) illustrate the effect of using a simplified criterion to determine whether a collision 
occurs. The actual state trajectories are projected with perfect accuracy, but the aircraft are 
represented by circles. In (a), the circles are larger than the aircraft, and in (b) they are 
smaller. In (a), a collision is predicted because the circles intersect, even though no collision 
is imminent In (b), the simplified criterion results in a failure to predict an imminent 
collision. Thus, even if the trajectories of all state variables are extrapolated with perfect 
precision, a disagreement between the notional collision and the implemented collision 
criterion can result in a false alarm or a missed detection. Generally, the collision criterion 
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is chosen so that alerting failures due to it are all false alarms (e.g. the sphere defining the 
aircraft encloses the aircraft entirely). 

State Trajectory Errors 

The same set of variables chosen to define the collision must be projected from initial 
measured values into the future by some dynamic or kinematic model. For aircraft that may 
be blundering (i.e., are flying unpredictably), this can be done accurately for only a limited 
interval. In Figure 3.2 (c) and (d), a prediction logic is shown that perfectly captures the 
meaning of a collision, but suffers state errors with increasing time. (In general, errors may 
exist at the initial time as well.) The actual state trajectories are represented by solid lines, 
and the modeled trajectories by dashed lines. In (c), a collision is predicted to occur when 
no collision is imminent because of errors in the predicted state variables. In (d), an 
imminent collision is misjudged as a miss due to trajectory errors. 


Collision Simplification Error State Trajectory Error 



Figure 3.2: Two Types of System Modeling Error 
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The useful time range of a dynamic model is affected by the number and choice of 
state variables. The most primitive means of projecting the location of an aircraft along a 
single trajectory is to assume constant velocity flight. Because any parallel approach 
blunder must involve at some point a turn away from the runway centerline, assuming the 
aircraft is initially established on a normal approach, a decision was made to include turn 
rate in the trajectory prediction model. This probably improves short term prediction of the 
aircraft position over the constant velocity model, provided reasonably small measurement 
errors occur, and may improve prediction over a longer interval if blunderers are indeed 
likely to follow coordinated turning paths for several seconds at a time. It does not follow 
that any addition of state variables to the model will necessarily improve prediction. For 
example, higher derivatives of positional variables may vary sharply over short time 
intervals, making them poor predictors over the interval of interest. The group of variables 
chosen for the AILS model, including position, velocity, and turn rate, is likely as complete a 
set as is reasonable with the current understanding of blunders. 

The AILS state variables are not sufficient for precise trajectoiy prediction over the 
entire time range of interest. A single-trajectory prediction based on these would result in 
both false alarm and missed detection alerting failures + . Due to the relatively high cost of 
collisions, it is in the interest of the system designer to bias the overall failure rate in favor of 
false alarms, and hopefully toward unnecessary alerts. It was mentioned before that careful 
choice of the collision criterion can achieve this. Another method employed by AILS is to 
expand the trajectory prediction to cover a range of possibilities, and to assume the worst 
case among these. It is equivalent to think of this as a modified single-trajectory model in 
which the intruder exhibits an intent to produce a collision. This will result in a higher 
overall alert rate, a higher false alarm rate (assuming the intent assumption is not generally 
correct), and consequently a higher induced collision rate, but fewer missed detections and 
late alerts. For this modification to be worthwhile, the increase in induced collisions must 
be less than the decrease in missed detections and late alerts. 


Such a logic was tested by Rockwell-Collins. See Koczo 1996. 
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Prediction Cutoff Time 


When the accuracy of state predictions is known to degrade with time, it is desirable 
to delay alerts to reduce false alarms. At the same time, alerts must take place early enough 
that time remains to avoid collisions. An appropriate time threshold might be determined by 
projecting the evasion as well as the nominal trajectory at each instant in time, and alerting 
only when the evasion maneuver is about to become unsafe. Because there would be errors 
in predicting the evasion maneuver as well as the nominal trajectory, some false alarms 
might induce collisions, and some collisions predicted early enough to be prevented with a 
perfecdy executed evasion maneuver might not actually be prevented. 

AILS does not use precisely the above method, but one less computationally involved. 
Rather than compute the time threshold at each moment, it issues an alert a fixed time before 
the predicted collision. The value of this constant is chosen to optimize the average 
performance of the system. It is clear that if one were to consider a variety of imminent 
collision scenarios, the time prior to collision at which the evasion would need to begin 
would not be the same in every case. For example, suppose the evasion maneuver for a host 
aircraft on approach is a vertical acceleration (“pull-up”) ending in a specified climb rate. 
Depending on the initial vertical speed of the host, execution of such a maneuver will involve 
an acceleration of different duration, produce a different change in velocity, and will 
therefore take a different amount of time to achieve the minimum displacement from the 
nominal trajectory needed to avoid a collision. With its fixed time threshold parameter, 
AILS suffers additional late alerts and/or false alarms. Which type of failure is 
predominant will depend on the chosen value of the time parameter. A large value favors 
false alarms and a small one favors late alerts. 

The chosen value of the time parameter should depend on the form of the planned 
evasion maneuver. A maneuver providing a quicker divergence from the nominal path 
would allow later alerting prior to a collision, and therefore a smaller time parameter value. 
Because this would limit trajectory prediction to a range where state error is lower, the 
overall rate of alerting failures could be expected to decrease with increasing maneuver 
aggressiveness. 

Summary of AILS Failure Causes 

In summary, a certain rate of alerting failures are unavoidable for a number of reasons 
that were discussed. Viewed relative to the structure of a fictitious ideal alerting system, 
properties of the AILS logic can be related to the logic’s tendency to favor some categories 
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of failure over others. These categories include missed detections, late alerts, unnecessary 
alerts, and induced collisions. To an extent, alerting failures can be reduced through 
intelligent design, but ultimately there will be some rate of failures. The tradeoff between 
different types of failures can be manipulated by varying the collision criterion, the model 
that projects aircraft states from initial values, the time to alert prior to an encounter, and the 
form of the evasion maneuver. 

3.4 SOC Curve Analysis 

A System Operating Characteristic (SOC) curve is a plot showing the tradeoff of 
false alarms for collisions (Kuchar 1996). The probabilities of collision and false alarm 
failures are plotted against one another for the multiple aircraft system as one parameter of 
the alerting system or operating environment is varied. The main purpose of SOC curves in 
this research is to compare the potential benefit of the alerting system under different 
evasion maneuvers. A detailed development of SOC curves in the context of trajectory 
simulation follows. 

The failures of an alerting system were described in section 3.2. An alerting system 
can also succeed, of course, by issuing an alert that prevents a collision that would occur 
otherwise (a correct detection ), or by refraining from alerting when no collision is imminent 
(a correct rejection). Table 3.1 presents a set of alerting outcomes that is mutually 
exclusive and exhaustive, and includes both failures and successes. Note the inclusion of 
two-letter abbreviations for each event. False alarms include both unnecessary alerts (UA) 
and induced collisions (IC), and collisions include missed detections (MD), induced 
collisions, and late alerts (LA). 

In this research the trajectories of pairs of aircraft were simulated, as influenced by the 
AILS alerting logic, and the outcome was classified using the above outcome set. For the 
SOC analysis it was necessary to estimate probabilities of different events. This was done 
on a relative frequency basis, using an assumption that all simulated trajectory pairs 
involving breakout alerts were of equal probability. 


33 


Table 3.1: Mutually Exclusive Alerting Outcomes 




Collision would occur 

Alert is 

Collision does 



without alert? 

issued? 

occur? 

Correct Rejection 

CR 

no 

no 

no 

Missed Detection 

MD 

yes 

no 

yes 

Unnecessary Alert 

UA 

no 

yes 

no 

Induced Collision 

IC 

no 

yes 

yes 

Correct Detection 

CD 

yes 

yes 

no 

Late Alert 

LA 

yes 

yes 

yes 


Two probabilistic quantities are required. One is the probability that a breakout alert 
is a false alarm. It is termed P(FA), and is defined as 


P(FA) = 


UA + IC 

MD + UA + IC + CD + LA 


( 3 . 1 ) 


The denominator of this expression is the total number of breakout alert events. A MD 
trajectory scenario is classified as a breakout event, even though technically the collision 
occurs before an alert is issued. In effect, the collision is the alert. The numerator is the 
total number of false alarm events. 


The second quantity is P(SA), the probability that a breakout alert has a successful 
outcome (SA stands for “Successful Alert”). A success is any breakout alert outcome 
other than a collision. Thus, 


P(SA) = 1 - 


MD + IC + LA 

MD + UA + IC + CD + LA 


( 3 . 2 ) 


where the denominator of the second term is the same as before, and the numerator is the 
total number of collision events. An equivalent expression is 
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or 


P(SA) = 


MD + UA + IC + CD + LA MD + IC + LA 

MD + UA + IC + CD + LA MD + UA + IC + CD + LA 


P(SA) = 


UA + CD 

MD + UA + IC + CD + LA 


(3.3) 


To construct an SOC plot, P(SA) is plotted with respect to P(FA) for the alerting 
system and operating environment of interest. The alerting system maps each possible 
value of P(FA) into a single value of P(S A). As a parameter of the overall system is varied, 
the system operating point traces out an SOC curve, illustrated in Figure 3.3. 



Figure 3.3: System Operating Characteristic Curve 

An intuitive quality of a beneficial alerting system is that, in the course of a trajectory 
scenario, a collision is less likely to occur if the system chooses to alert than when such an 
alert does not occur. If a collision is equally likely to occur whether the alert occurs or not, 
there is no benefit. If a collision is more likely when the alert occurs, the alerting system is 
causing harm. 

P(SA) is the probability that no collision will occur given that an alert occurs, or P(no 
collision I alert). This is the same as 1 - P(collision I alert). P(FA) is the probability that a 
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given alert is a false alarm, or P(false alarm I alert). This is the same as P(no collision I no 
alert), which is the same as 1 - P(collision I no alert). Thus, P(SA) = P(FA) would imply 
that P(collision I alert) = P(collision I no alert), which is the condition for zero benefit 
described in the previous paragraph. P(SA) > P(FA) would imply that P(collision I alert) < 
P(collision I no alert), or that there is a benefit, and P(S A) < P(FA) would imply a harmful 
alerting system. 

In Figure 3.3, the line of zero alerting benefit is represented by a dashed diagonal line. 
Points above and below the diagonal correspond to beneficial and harmful alerting system 
configurations respectively. 

An ideal alerting system would produce no false alarms, and would successfully avert 
any impending collision, which would place the system operating point at the upper left 
comer of the SOC axes as shown. This condition is generally not reachable. As discussed 
in previous sections of the chapter, the alerting system may be adjustable in various ways, 
but no combination of system parameter values will yield perfect performance. 

An example SOC curve illustrates that, generally, as P(FA) increases, P(SA) does as 
well. The maximum possible value of P(FA) is less than 1. By inspection of expression 
3.1, the maximum P(FA) occurs when the sum UA + IC is maximum. The sum MD + CD 
+ LA is the total number of imminent collision scenarios, termed ICS, and is fixed by the 
choice of trajectories. UA + IC is maximum if all remaining scenarios produce false 
alarms. Thus, if there are N trajectory scenarios in total. 


max[P(FA)] = 


UA + IC 
ICS + (UA + IC) 


N - ICS ICS 

ICS + (N - ICS) N 


(3.4) 


There is no inherent limit on the range of P(SA). It may be driven to 1 if the ratios of 
IC over UA + IC and of MD + LA over MD + CD + LA go separately to 0 (or identically, 
if IC and MD + LA go to 0). Note that in the first ratio an increase in UA has the same 
effect as a reduction in IC. This means that an increase in induced collisions can be 
obscured by an increase in unnecessary alerts if the total of false alarms is allowed to 
change. P(S A) therefore should not be viewed as a measure of safety independent of 
P(FA). P(SA) will increase with increasing P(FA) even if the collision rate remains 
constant But for a fixed P(FA), differences in P(SA) indicate relative benefit. This is 
because any increase in P(SA) at constant P(FA) implies either a reduction in IC or an 
increase in CD, both of which are desired. 
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Chapter 4 


Analysis 


4.1 Overview of Analysis 

The goal is to compare the alerting performance possible with each of the two 
candidate evasion maneuvers under the AILS algorithm. Using a variety of simulated 
aircraft trajectories, the failure rates of AILS are evaluated over a range of values of logic 
parameters. Specifically, the T and R parameters, which are analogous to projection time 
and the size of the simplified aircraft, are the focus of attention. Host pilot reaction time is 
also varied, to determine the effect of increasing delay on the success of each maneuver. 
Failure totals are viewed directly as functions of the controlled parameters, and are also used 
to generate SOC curves, allowing a qualitative understanding of the benefit of each 
maneuver. 

Four runway spacings were tested, but due to the limitations imposed on independent 
approach runway spacing by wake vortices, the current runway separation goal for AILS is 
2500 ft. The numbers discussed in this chapter are thus for the 2500 ft case. Other data are 
included in Appendix A. 

4.2 Trajectory Simulation 

Alerting performance data were generated through computer simulation of a large 
number of parallel approach trajectory scenarios. Two aircraft were simulated in each 
scenario, which always began with both established on final approach along adjacent 
extended runway centerlines, and located vertically on a 3° glide slope. One aircraft, referred 
to as the host, behaved as though equipped with the AILS alerting system. This aircraft 
performed an ideal normal approach, at a constant 145 kt airspeed and with no bank or track 
angle variation about the nominal values. The other aircraft, called the intruder, followed a 
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variety of blunder and normal approach trajectories, one for each scenario. If blundering, 
the intruder behaved as though unable to respond to any alerts that would have been issued 
by its own AILS alerting system. 

When issued a breakout alert by the AILS logic, the host aircraft performed a specific 
evasion maneuver. If the centers of the two aircraft passed within 500 feet of one another in 
the course of a scenario, each lasting about 2 minutes, they were said to have collided. If an 
alert was issued and an evasion took place, the host aircraft’s nominal trajectory was earned 
through as well so that the non-alert outcome could be determined. The outcomes of each 
scenario were recorded for later analysis. 

4.2.1 Rockwell-Collins Horizontal Trajectory Set 

Because no data exist for blunders occurring during GPS-based parallel approaches, 
the blunder scenarios used in this simulation are speculative. A set of 39 pre-recorded 
trajectories covering a variety of hypothetical approach behaviors, including both normal 
approaches and blunders, form the basis of the MIT simulation. The trajectories were 
created at Rockwell-Collins using a Fokker 70 part task simulator (Koczo 1996). They 
were designed to cover a variety of intuitive blunder types. Several blunder attributes were 
recognized and varied to produce the trajectories, including nominal airspeed, horizontal 
blunder form, and wind conditions. 

The Rockwell trajectories were used only to model the horizontal behavior of the 
intruder, though they were recorded in 3 dimensions. Desired vertical behavior was 
generated as described in section 4.2.2. 

Airspeed 

Three no min al airspeeds were used: 130, 145, and 160 knots. The second is an 
approximate average approach speed for large jet aircraft, and the first and last provide some 
spread about this value. 

Horizontal Trajectory Form 

All Rockwell trajectories began with the aircraft set up on correct approach along the 
runway centerline. After following the centerline for approximately 15,000 feet, the intruder 
performed one of the following seven maneuvers: a coordinated turn onto a new heading; a 
continuing constant bank turn; an "overadjust" blunder, where the intruder drifted in a 
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Lateral Position (ft) Lateral Position (ft ) Lateral Position (fi) 


30° Heading Blander 15* Heading Blunder 



5° Heading Blunder Constant 5* Bank Blunder 



Distance From Touchdown (ft) Distance From Touchdown (ft) 



Normal Approach 

7500 i 



Distance From Touchdown (ft) 


Figure 4.1: The Seven Horizontal Intruder Trajectory Forms 
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Bank Angle (deg) 


direction opposite the host’s centerline, and then overshot in recovering; a "fake" blunder, 
where the intruder drifted toward the host’s centerline but corrected; and finally a normal 
approach, where no significant deviation from the intruder centerline occurred. For the 
heading change blunder the intruder made a coordinated, standard-rate turn toward the 
host’s centerline, rolling out on a new heading 5°, 15°, or 30° away from the approach 
heading. For the constant bank blunder the intruder rolled into and maintained a 
coordinated turn at a 5° bank for the duration of the blunder. The fake and overadjust 
blunders adhered to no precise numerical specification. In all there are seven horizontal 
intruder trajectory forms. Representative Rockwell trajectories for each of the seven are 
illustrated in Figure 4. 1 . 

Wind Conditions 

Because the Rockwell trajectories were flown by humans they contain normal flight 
technical errors. In addition, two wind conditions were simulated, including calm air and 
what is referred to in the Rockwell report (Koczo 96) as “moderate turbulence,” with 12.5 
kt gusts in three dimensions. Thus, trajectories flown in moderate turbulence contain 
frequent spikes in bank angle and other states. Figure 4.2 illustrates the effect of wind 
gusts on bank angle during a normal approach. 


With Wind Gusts Without Wind Gusts 
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Figure 4.2: Effect of Wind Gusts on Bank Angle 
(Normal Approach Trajectories) 

The product of all described attributes (airspeed, horizontal trajectory form, and wind) 
is 42. However, only 39 final trajectories were used because 3 attribute combinations were 
not available in the provided trajectory set, namely the overadjust trajectories for gusty 
conditions. 

States along each Rockwell trajectory were sampled at intervals of 0.4545 seconds, or 
about 2 Hz. Though 1 Hz is perhaps a more realistic estimate of datalink capability, in the 
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MIT simulation the AILS logic was provided states at the 0.4545 second update rate of the 
trajectory files. 

No filtering was performed on the states provided to the alerting logic. 

4.2.2 Vertical Augmentation of Horizontal Trajectories 

The horizontal trajectories were augmented with a range of vertical maneuvers at run 
time by the simulation program. Prior to any blunder, the host adhered to a 3° glide slope. 
Simultaneously with the start of the horizontal blunder the intruder either remained in the 
plane of its glide slope, or accelerated vertically at approximately 0.25 g to level flight or to a 
constant climb at 500, 1000, 1500 or 2000 feet per minute (Figure 4.3). This augmentation 
was performed only if the horizontal trajectory contained some type of blunder. Normal 
approach trajectories were simply repeated without modification. 



2000 fpm 
1500 fpm 
1000 fpm 
500 fpm 
level 

glide slope 


Figure 4.3: Range of Intruder Vertical Behavior 

Some liberty was taken in assuming that lateral and vertical dynamics are sufficiently 
independent that states such as bank and airspeed, which are contained in the horizontal 
trajectory file, are unaltered over a range of vertical behaviors. Given the small magnitude of 
vertical speeds relative to lateral, this is probably a harmless assumption. 

The combination of 6 vertical and 39 horizontal trajectory cases resulted in 234 
distinct intruder trajectories. 
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4.2.3 Variation of Initial Relative Position 

In each scenario one of the 234 intruder trajectories was paired with a 145 kt normal 
approach host trajectory. To ensure a variety of encounters, the relative initial position of 
the intruder with respect to the host was varied both vertically and longitudinally. Initial 
lateral separation was fixed according to the runway separation. 

Longitudinally (in a direction parallel to the runway centerlines), the initial position of 
the intruder was varied between 8900 ft in front of and 9100 ft behind the host, in 
increments of 600 ft. This is approximately a 3 nmi range. Vertically, the initial position of 
the intruder was varied between 1000 feet above and 1000 feet below the host, in 500 foot 
increments. Four runway spacings, 1700 feet, 2000 feet, 2500 feet, and 3400 feet, were 
examined. 

In all there were (31 horizontal) x (5 vertical) = 155 initial relative positions per 
runway spacing. Recalling that there were 234 intruder trajectories, the total number of 
relative intruder trajectories per runway spacing was 155 x 234 = 36270. 

4.2.4 Format of Output Data 

For each run of the program, the host aircraft was subjected to the 36270 scenarios at 
each of four different runway spacings. The outcome of each scenario was classified 
according to the six mutually exclusive categories listed in Table 3.1. Outcomes were 
distinguished by whether or not an alert was duly needed to prevent a collision (determined 
by observing the non-evasion host trajectory), whether an alert was issued by the AILS 
system, and whether a collision did occur when the host pilot followed whatever advice was 
given by the alerting system. Totals of the six outcomes were provided for each Rockwell- 
Collins horizontal trajectory at each runway spacing. Summing over all trajectories for each 
runway spacing provided the input values of CD, UA, IC, MD, and LA used to generate 
SOC curves. For other analysis, the summation was performed separately over normal 
approach and blunder trajectory types. 
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4.3 AILS System Parameter Variation 

Threshold Parameters 

Two parameters of the alerting threshold were varied: R, the horizontal range 

parameter, and T, the projection time parameter. The vertical range parameter, H, was fixed 
at 550 fit in all simulations. R was varied from 350 to 750 ft in 100 ft increments. T was 
varied from 5 to 25 seconds in 2 second increments. 

Evasion Maneuver 


Two evasion maneuvers were tested: a standard tum-climb as assumed in the original 
AILS design, and a simple climb-only along the runway centerline. The tum-climb includes 
a pilot latent interval followed by a 45° coordinated heading change with a 30° maximum 
bank, a 0.25 g pull-up to a 2000 ft/min climb, and a 15 kt airspeed increase at 1 kt/s. The 
climb-only maneuver is modeled as the latent interval followed by the pull-up and airspeed 
increase, with the aircraft continuing to follow the extended runway centerline. 

Pilot Reaction Time 

The reaction time (or latent interval) of the host aircraft pilot was varied from 2 to 1 1 
seconds in 3 second increments. 

4.4 Blunder vs. Normal Approach Intruder Trajectories 

It was mentioned in section 4.2.1 that the chosen set of blunders is hypothetical, and 
therefore data resulting from their use can only be interpreted loosely. Even if the makeup 
of blunders were accurate, there is another difficulty with the simulation. Because the 
majority of tested intruder trajectories were blunders, a probabilistic interpretation of all 
trajectories as equally likely will not be appropriate for estimating overall safety levels or 
false alarm rates. In reality, blunders will probably occur at an extremely low rate compared 
with normal approaches. Suggested per-approach probabilities of an approach blunder in a 
well-designed system are in the neighborhood of 10 7 (Kelly & Davis 1994) to 10 s (PRM 
Program Office 1991). Because there is no definite knowledge of the frequency with which 
blunders will occur in future parallel approach systems, and because operating a simulation 
with blunders occurring in correct proportion to normal approaches would require a 
prohibitively long run time, the simulation used for this research makes no attempt to be 
proportionally accurate. 
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Unfortunately, the simulation output has little value unless some probabilistic 
meaning can be attributed to it. In compromise, normal approaches by the intruder are 
analyzed separately from the blunders in all but the SOC type of plot presented in this 
chapter. Each of the two sets is treated as a sample space, with each element equally likely. 
The outcome rates for each set are then interpreted as probabilities, conditional on the 
intruder following that type of trajectory. In the SOC analysis, all data are lumped together. 

4.5 Results 


4.5.1 False Alarms 


Whether or not a false alarm occurs does not depend on the particular maneuver used, 
so these results apply equally to both evasion maneuvers. The quantity “false alarm 
fraction” is the fraction of all trajectories of a given subset (blunder or normal approach) 
with a particular combination of R, T, reaction time, and runway spacing, for which a false 
alarm occurred. In terms of the outcome categories introduced in chapter 3, it is given by 


False Alarm Fraction = 


UA + IC 

MD + UA + IC + CD + LA + CR 


(4.1) 


Figure 4.4 summarizes false alarm data over the full range of R and T. Larger values 
of either R or T tend to produce a larger false alarm fraction. For the blunder subset, false 
alarms are impossible to eliminate with any combination of R and T. As might be predicted, 
the lowest false alarm rate achieved is with the smallest values of R and T (350 ft and 5 sec 
respectively). This is equivalent to minimizing the size of the modeled aircraft, and 
assuming a high performance evasion maneuver that requires little advance warning to 
execute. For the normal approach subset, the false alarm fraction is zero for low values of 
T, though false alarms can be made to occur with sufficiently large values of R and T. It is 
likely that normal approach false alarms would occur for smaller R and T values if a more 
complete set of normal approach trajectories were used. 
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Figure 4.4: False Alarm Fraction 


It is useful to partition false alarm data into mutually exclusive components of 
unnecessary alerts and induced collisions, because the cost ambiguity of a false alarm 
makes it a poor metric of performance. Because the rate of induced collisions depends on 
the actions of pilots following alerts, unnecessary alerts are also a function of the evasion 
maneuver. Figure 4.5 illustrates the relative tendency of each evasion maneuver to induce a 
collision following a false alarm. The vertical range is the fraction of false alarms producing 
induced collisions. In terms of the mutually exclusive outcome categories, this is 

IC/FA = — (4.2) 

UA + IC 


Thus, 1 - IC/FA is the fraction of false alarms that are unnecessary alerts, and may also be 
read directly from the plots. Pilot reaction time is 2 sec for this data. Normal approaches 
are not included, because induced collisions were never observed to result from normal 
approach false alarms. In other words, all false alarms for normal approach trajectories 
were unnecessary alerts, and may be read directly from Figure 4.4. 
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Figure 4.5: Fraction of Blunder False Alarms Inducing Collisions 

For low to intermediate values of T, induced collisions make up a significant fraction 
of false alarm cases when using the climb-only as compared with the turn-climb. Because 
any false alarm is the result of an error in trajectory prediction. Figure 4.5 suggests that the 
success of alerting is less sensitive to errors in prediction with the tum-climb than with the 
climb-only. IC/FA falls off for large T regardless of the maneuver, but never reaches 
negligible levels with the climb-only in the tested range of T. IC/FA should fall off for large 
T, because the overall false alarm rate (denominator) increases, and because the earlier a 
false alarm occurs, the larger the separation at the time the evasion begins, and the less likely 
it is that a given maneuver will produce a collision. 

At nominal threshold values of R = 550 ft and T = 13 sec, the climb-only IC/FA ratio 
is 0.250, about 40 times the tum-climb ratio of 0.00655. 

With the version of AILS assumed here, once an alert is issued, all monitoring of the 
situation ceases. The evasion maneuver is an open loop procedure. The final version of 
AILS could include adaptive vertical guidance, such as implemented in TCAS, which could 
improve the failure performance of either maneuver. 

4.5.2 Collisions 


The collision fraction is the fraction of all blunder trajectories that for any reason 
resulted in a collision at the given system settings. It is given by 


Collision Fraction = 


MD + IC + LA 

MD + UA + IC + CD + LA + CR 
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Figure 4.6: Collision Fraction for Blunder Trajectories 

The blunder trajectory collision fraction for both evasion maneuvers, all combinations 
of R and T, and a 2 sec pilot reaction time is shown in Figure 4.6. The collision fraction 
depends strongly on the form of the evasion maneuver. Whereas with the tum-climb 
maneuver collisions can be virtually eliminated with certain combinations of R and T, this 
same feat is impossible with the climb-only. To obtain the lowest possible collision fraction 
with the climb-only, large values of R or T are required, while with the tum-climb a low 
collision fraction can be obtained with mid-range choices of R and T. This is important in 
view of the fact that the rate of false alarms is minimum for small values of the two 
threshold parameters. It may therefore be easier to reach a desirable compromise between 
collisions and unnecessary alerts using the tum-climb maneuver. 

At nominal threshold values of R = 550 ft and T = 13 sec, the climb-only collision 
fraction is 0.00499, about 40 times the tum-climb collision fraction of 0.000131. 

4.5.3 SOC Representation of Threshold Variation Effects 

Figure 4.7 shows SOC curves for three values of the R parameter as T is varied from 
5 to 25 seconds in 2 second increments. Pilot reaction time is fixed at 2 sec. For T = 5 sec 
the operating point is on the left end of each curve. As T increases, the false alarm rate 
tends to increase. For a given T, small R values result in the lowest false alarm rate, but also 
the lowest rate of successful alerts. 

Recalling the meaning of the dashed diagonal line as an indicator of alerting benefit, it 
is appears that the climb-only with any threshold setting can be surpassed by the tum-climb 
in benefit Regardless of the threshold setting, alerting with the tum-climb provides some 
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benefit. As parameters are varied with the climb-only, alerting benefit changes little, and the 
operating point remains in the neighborhood of the diagonal, sometimes even falling below. 
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Figure 4.7: SOC Curves for Varied T at Three R Values 


4.5.4 Host Pilot Reaction Time 

There is concern that with the expected rarity of alerts in a well operating parallel 
approach system, the reaction time of the host pilot may be longer than the assumed 2 
seconds. To see the approximate effect of response time on the collision rate, the collision 
fraction is plotted in Figure 4.8 as a function of reaction time. This data is for blunder 
trajectories and the nominal parameter settings for a 2500 ft spacing: R = 550 ft and T = 13 
sec. Although collisions increase with reaction time for either maneuver, there are notable 
differences. For the climb-only the rate of increase is initially higher than for the turn- 
climb, then slows. The rate of increase grows for the turn-climb as reaction time increases. 
By 1 1 seconds, the collision fraction for the tum-climb has increased approximately 0.75 as 
much as for the climb-only. Overall, the tum-climb is superior in its resistance to 
degradation with increasing reaction time, particularly within a few seconds of the assumed 
2 sec value. 
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Figure 4.8: Collision Fraction vs. Reaction Time (R = 550 ft, T = 13 sec) 
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Figure 4.9: SOC Curves for Varied T at Two Reaction Times 


In Figure 4.9 the effects of increasing reaction time are shown using SOC curves. 
Again, T is the parameter varying along the curve, but now each curve represents a different 
reaction time. In the interest of minimizing clutter, curves for only the 2 and 1 1 sec reaction 
times are shown. The 5 and 8 sec curves lie predictably between these for the most part. As 
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reaction time increases, the probability of a successful alert decreases while the probability 
of a false alarm remains constant. For mid-range values of T in the neighborhood of the 13 
sec nominal setting, the SOC curve for the turn-climb maneuver moves toward the diagonal, 
but retains some benefit even at an 1 1 second reaction time. When using the climb-only the 
benefit is slight to begin with, and with increasing reaction time nearly the entire SOC curve 
drops below the diagonal. 
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Chapter 5 


Summary and Conclusions 


5.1 Research Summary 

An airborne collision avoidance alerting system may improve the safety of closely 
spaced parallel approaches. Upon prediction of a collision, the system would issue an alert, 
allowing just enough time for the pilot of an endangered aircraft to execute a trained evasion 
maneuver. Unavoidably, such an alerting system would suffer occasional failures, including 
missed detections, late alerts, unnecessary alerts, and induced collisions. Though failures 
can not be eliminated, their frequency and proportions can be manipulated to some extent 
through adjustment of the alerting threshold. Threshold adjustment is a necessary task 
during initial design, and must be repeated at any time a significant change (such as the 
choice of evasion maneuver) is made to the alerting system or operating environment. 

Different evasion maneuvers will have different performance potential in terms of 
safety and false alarm rate, but operational expense and complexity are also relevant in 
choosing between maneuvers. 

Two candidate evasion maneuvers for parallel approach collision avoidance are the 
turn-climb and climb-only. The climb-only maneuver is operationally preferable, but would 
sacrifice performance relative to the turn-climb. An analysis was performed to determine 
the extent of the performance loss associated with the climb-only, and whether or not it 
could be a reasonable substitute for the established turn-climb. 

Simulations of hypothetical approach and blunder scenarios were run to gain a rough 
understanding of the performance obtainable with each of the two evasion maneuvers. 
Alerts were triggered aboard a simulated host aircraft using an experimental alerting logic 
that was developed by Rockwell-Collins and the NASA Langley Research Center. The 
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alerting threshold was adjusted by varying two parameters of the logic. To examine the 
sensitivity of performance to pilot reaction time, a reaction time parameter of the evasion 
maneuvers was varied as well. 

Performance metrics were plotted as functions of the threshold and maneuver 
parameters. By partitioning data into normal approach and blunder scenario types, and 
assuming uniform probability within each of the two groups, approximations could be made 
of the threshold settings yielding the best performance for each maneuver. Relative 
performance potential was also apparent in the plots. 

System Operating Characteristic curves were generated by lumping all scenario types 
together and plotting the false alarm versus the collision probability for varied threshold and 
maneuver parameter settings. These provide a qualitative understanding of the benefit of the 
alerting system with different evasion maneuvers. 

5.2 Conclusions 

A climb-only evasion maneuver of the magnitude assumed in this research appears to 
be a poor choice, even if more convenient operationally. At nominal threshold parameter 
values of R = 550 ft and T = 13 sec, and assuming a 2 sec pilot reaction time, false alarms 
during blunders were approximately 40 times as likely to induce collisions with the climb- 
only, and the climb-only collision rate for blunder trajectories was 40 times as high overall. 
The SOC curve analysis shows that at nominal parameter values the tum-climb gives a 
pronounced safety benefit. At the same threshold setting, the benefit of the climb-only is 
small, with the operating point located only slightly above the diagonal. Based on the shape 
of the SOC curves, it is not clear that adjustment of the threshold parameters away from 
nominal values will yield a notable improvement in climb-only performance. 

If pilot reaction time is increased over the nominal 2 sec value, both maneuvers suffer 
a penalty in benefit, but even at an 1 1 sec reaction time the benefit of the tum-climb exceeds 
that of the climb-only at a 2 sec reaction time. The climb-only SOC curve is quickly driven 
below the diagonal as reaction time increases. Thus, tum-climb safety seems to be less 
sensitive to variation in reaction time, and perhaps this quality extends to other types of 
execution error. 

Some aspects of the experimental method used are questionable enough to deserve 
further discussion, which follows. 
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The modeled set of blunder trajectories was hypothetical. The little anecdotal and 
recorded information available regarding blunders that have occurred does not allow 
construction of a high fidelity blunder model. If it were known that most blunders will be 
of a particular variety, and the simulation were based on this knowledge, significant changes 
could occur in the observed performance of either maneuver, perhaps making the climb-only 
a reasonable option. Due to a lack of such detailed knowledge, numerous blunder types 
were considered. Unfortunately, at some point a weight must be assigned to each type of 
blunder, and in assuming equal probability in the analysis of data, it is possible that a 
maneuver’s poor performance in handling one type of blunder can be obscured by good 
performance in another type of scenario. 

There is also a question of whether the chosen set of trajectories adequately covers the 
actual range of possibilities. The blunders in the chosen set are fairly simple, and it is easy 
to imagine more complex blunders that are possible, but are excluded. For example, it 
would be reasonable to assume that an intruder might modify a blunder trajectory in 
response to an AILS alert, perhaps initiating an evasion maneuver or attempting to resume a 
normal approach. In all trajectories used, the intruder follows a predictable, usually constant 
velocity, trajectory after the initial blunder. Therefore, the assumption that an intruder fails 
to respond to alerts may not ensure the “worst case” after all. 

The actions of air traffic control are not included in the simulation model. In the 
simulation, a trajectory scenario ends when the end of the pre-recorded trajectories are 
reached. These are approximately 2 minutes in duration, with the blunder occurring early 
on in this interval. Air traffic control is likely to intervene within a fraction of a minute of 
when a blunder begins, meaning that some collisions that occurred during the simulation 
might have been prevented if ATC had been modeled. This may be an important 
consideration if there are blunder scenarios in the simulation that produce a slow 
convergence between the intruder and host even after performance of the nominal evasion 
maneuver (e.g., a large-angle heading blunder by a fast intruder, in combination with the 
tum-climb evasion), and with slow drift blunders, where plenty of time should exist for ATC 
intervention before an AILS breakout alert even occurs. 

An issue that has been studied in other parallel approach alerting research (Pritchett & 
Hansman 1997), but was avoided in design of the trajectory simulation used here, is the 
relationship between the rate of normal approach false alarms (unnecessary alerts) and pilot 
conformance to breakout alerts. An alerting system that generates unnecessary breakout 
alerts at too high a rate may lose the confidence of pilots, who begin to respond erratically to 
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breakout alerts, whether they are legitimate or not. This will generally have a harmful effect 
on collision prevention. 

In plotting data from the simulation, an implicit assumption was made that the pilot 
adhered perfectly to any breakout alert. Because of the difficulty of designing a true Monte 
Carlo simulation, where normal approaches and blunders occur at the correct relative 
frequencies, it was impossible to model the dependence of pilot behavior on unnecessary 
alerts. With this in mind, special care must be taken in interpreting the data. Safety data 
corresponding to threshold parameter combinations that can result in frequent unnecessary 
alerts during normal approaches (Given the rarity of any blunder, normal approach 
unnecessary alerts should make up the vast majority of unnecessary alerts.) must be deemed 
unusable. Referring to Figure 4.4, this means that T must be limited to values below around 
19 sec (depending on the choice of R), because above this the rate of normal approach false 
alarms increases suddenly to above zero. Note that because normal approaches were not the 
focus of this research, the selection of normal approach trajectories was small, and 
consequently the normal approach false alarm data may be optimistic. A more complete 
probabilistic selection of normal approach trajectories would result in a measurable rate of 
false alarms at smaller values of T. 

Because performance data for the full range of T is included in the SOC plots, it is 
important to avoid setting the operating point within the region of large T, where the true 
system behavior is unknown. 

A final item to note concerning the simulation is its discrete method of trajectory 
“sampling.” Output was interpreted as though generated by a Monte Carlo simulation in 
which several variables were uniformly and continuously distributed. In reality, each initial 
state variable was limited to discrete values determined by the chosen range and increment. 
This may account for such phenomena as the apparent ability of the climb-only to have a net 
harmful effect on safety for certain values of R and T. It may also explain the tendency of 
increasing pilot latency to drive the system operating point below the diagonal, when 
intuition suggests that the effect of a very large reaction time should be to reduce benefit to 
exactly zero. 
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Appendix A 


Selected Simulation Results 


This appendix gives a more comprehensive sample of output from the simulation 
described in Chapter 4. Data are included for all four tested runway spacings: 1700 ft, 
2000 ft, 2500 ft, and 3400 ft. Both the SOC and event fraction types of plot are present. 

Relevant event fractions are defined as follows: 


False Alarm Fraction 


UA + IC 

MD + UA + IC + CD + LA + CR 


Unnecessary Alert Fraction 


UA 

MD + UA + IC + CD + LA + CR 


Collision Fraction = 


MD + IC + LA 

MD + UA + IC + CD + LA + CR 


Induced Collision Fraction = 

MD + UA + IC + CD + LA + CR 

In Figures A1 to A5 event fraction data for both maneuvers and all runway spacings 
are presented as functions of R and T at a 2 sec pilot reaction time. 

In Figures A6 and A7 collision and induced collision event data are presented as 
functions of T and reaction time at R = 550 ft. 

In A8 to A1 1, selected data are presented in SOC format. 
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Figure Al: False Alarm Fractions 
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Figure A2: Unnecessary Alert Fractions for Turn-Climb Maneuver 
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Figure A3: Unnecessary Alert Fractions for Climb-Only Maneuver 
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Figure A4: Induced Collision Fractions for Blunder Trajectories 
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Figure A5: Collision Fractions for Blunder Trajectories 
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Figure A6: Collision Fractions for Blunder Trajectories 
R = 550 ft. Varied Reaction Time 
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Figure A7: Induced Collision Fractions for Blunder Trajectories 
R = 550 ft. Varied Reaction Time 
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Figure A8: SOC Curves for Tum-Climb Evasion Maneuver, Varied R and T 
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Figure A10: SOC Curves for Turn-Climb Evasion Maneuver 
R = 550 ft, Varied Reaction Time 
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Figure A1 1 1 SOC Curves for Climb-Only Evasion Maneuver 
R = 550 ft. Varied Reaction Time 







REPORT DOCUMENTATION PAGE 

Form Approved 
OMB No. 0704-0188 

Public reporting burden lor this collection ot information is estimated to average 1 hour per response, including the time tor reviewing instructions, searching existing data 
sources gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other 
aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate forlnfomnation Operations and 
Reports 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188), 

1 . AGENCY USE ONLY (Leave blank) 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED 

August 2000 Contractor Report 

4. TITLE AND SUBTITLE 

Performance Evaluation of Evasion Maneuvers for Parallel Approach 
Collision Avoidance 

576-02-11-17 
NAG 1-1974 

6. AUTHOR(S) 

Lee F. Winder and James K. Kuchar 

7. pfeftfORMiNG ORGANIZATION NAME(S) AND ADDRESSES) — ““ — 

Massachusetts Institute of Technology 
Cambridge, MA 

8. PERFORMING ORGANIZATION 
REPORT NUMBER 

9. SPONSORING/MONltORING AGENCY NAME(S) AND ADDRESS(ES) 

National Aeronautics and Space Administration 
Langley Research Center 
Hampton, VA 23681-2199 

10. SPONSORING/MONITORING 
AGENCY REPORT NUMBER 

NAS A/CR- 2000-2 1 0099 

11. SUPPLEMENTARY NOTES 

Langley Technical Monitor: Marvin Waller 

12a. DIStRIBUTlON/AVAILABiLITY STATEMENT 
Unclassified-Unlimited 

Subject Category 3 Distribution: Standard 

Availability: NASA CASI (301) 621-0390 

12b. DISTRIBUTION CODE 


13. ABSTRACT (Maximum 200 words) 


Current plans for independent instrument approaches to closely spaced parallel runways call for an automated 
pilot alerting system to ensure separation of aircraft in the case of a "blunder," or unexpected deviation from the 
normal approach path. Resolution advisories by this system would require the pilot of an endangered aircraft to 
perform a trained evasion maneuver. 


The potential performance of two evasion maneuvers, referred to as the "turn-climb" and "climb-only," was 
estimated using an experimental NASA alerting logic (AILS) and a computer simulation of relative trajectory 
scenarios between two aircraft. One aircraft was equipped with the NASA alerting system, and maneuvered 
accordingly. Observation of the rates of different types of alerting failure allowed judgement of evasion 
maneuver performance. System Operating Characteristic (SOC) curves were used to assess the benefit of 


alerting with each maneuver. 


14. SUBJECT TERMS 

Closely spaced parallel approaches. Evasion Maneuver Evaluation 
Airborne Information for Lateral Spacing (AILS) 


it. SECURITY classification” 
OF REPORT 

Unclassified 


OF THIS PAGE 

Unclassified 


15. SECURI T Y CLASSIFICA T ION ' 
OF ABSTRACT 

Unclassified 


15. NUMBER OF PAGES 

72 


16. PRICE CODE 

A04 


20. LIMITATION 
OF ABSTRACT 

UL 


Standard l-orm 2S8 (Hev. 2-89) 
Prescribed by ANSI Std. Z-39-18 
298-102 


N5N 7540-01 -2S0-bbUU 








